Sums Of Independent Random Variables Research Articles

Limit theorems for a class of processes generalizing the U-empirical process

In this paper, we develop theory and tools for studying U-processes, a natural higher-order generalization of the empirical processes. We introduce a class of random discrete U-measures that generalize the empirical U-measure. We establish a Glivenko-Cantelli and a Donsker theorem under conditions on entropy numbers prevalent in the theory of empirical processes. These results are proved under some standard structural conditions on the Vapnik-Chervonenkis classes of functions and some mild conditions on the model. The uniform limit theorems discussed in this paper are key tools for many further developments involving empirical process techniques. Our results are applied to prove the asymptotic normality of Liu's simplicial median. We conclude this paper by extending Anscombe's central limit theorem to encompass randomly stopped U-processes, building upon its application to randomly stopped sums of independent random variables.

Total variation distance and compound poisson approximations for random sums

We derive new upper bounds for the total variation distance between compound Poisson distributions as well as between a random sum and a compound Poisson distribution, and as a result we also obtain upper bounds for the total variation distance between compound Poisson distributions and a sum of independent random variables. These bounds are generalizations and refinements of some well-known bounds in the literature. We also derive upper bounds for the total variation distance between negative binomial distributions of order k and between the negative binomial distributions of order k and compound Poisson distributions. Upper bounds for the total variation distances between the number of success runs of length k in binary Markovian trials and its limiting distributions for several enumeration schemes, are also given.

On Poisson Approximation

AbstractThe problem of evaluating the accuracy of Poisson approximation to the distribution of a sum of independent integer-valued random variables has attracted a lot of attention in the past six decades. Among authors who contributed to the topic are Prokhorov, Kolmogorov, LeCam, Shorgin, Barbour, Hall, Deheuvels, Pfeifer, Roos, and many others. From a practical point of view, the problem has important applications in insurance, reliability theory, extreme value theory, etc. From a theoretical point of view, the topic provides insights into Kolmogorov’s problem concerning the accuracy of approximation of the distribution of a sum of independent random variables by infinitely divisible laws. The task of establishing an estimate of the accuracy of Poisson approximation with a correct (the best possible) constant at the leading term remained open for decades. We present a solution to that problem in the case where the accuracy of approximation is evaluated in terms of the point metric. We generalise and sharpen the corresponding inequalities established by preceding authors. A new result is established for the intensively studied topic of compound Poisson approximation to the distribution of a sum of integer-valued r.v.s.

Random Sampling of Mellin Band-Limited Signals

In this paper, we address the random sampling problem for the class of functions in the space of Mellin band-limited functions B T , which are concentrated on a bounded cube. It is established that any Mellin band-limited function can be approximated by an element in a finite-dimensional subspace of B T . Utilizing the notion of covering number and Bernstein’s inequality to the sum of independent random variables, we prove that the probabilistic sampling inequality holds for the set of concentrated signals in B T with an overwhelming probability provided the sampling size is large enough.

The Emergence of the Normal Distribution in Deterministic Chaotic Maps.

The central limit theorem states that, in the limits of a large number of terms, an appropriately scaled sum of independent random variables yields another random variable whose probability distribution tends to attain a stable distribution. The condition of independence, however, only holds in real systems as an approximation. To extend the theorem to more general situations, previous studies have derived a version of the central limit theorem that also holds for variables that are not independent. Here, we present numerical results that characterize how convergence is attained when the variables being summed are deterministically related to one another through the recurrent application of an ergodic mapping. In all the explored cases, the convergence to the limit distribution is slower than for random sampling. Yet, the speed at which convergence is attained varies substantially from system to system, and these variations imply differences in the way information about the deterministic nature of the dynamics is progressively lost as the number of summands increases. Some of the identified factors in shaping the convergence process are the strength of mixing induced by the mapping and the shape of the marginal distribution of each variable, most particularly, the presence of divergences or fat tails.

F-Divergences and Their Applications in Lossy Compression and Bounding Generalization Error

In this paper, we provide three applications for f-divergences: (i) we introduce Sanov’s upper bound on the tail probability of the sum of independent random variables based on super-modular f-divergence and show that our generalized Sanov’s bound strictly improves over ordinary one, (ii) we consider the lossy compression problem which studies the set of achievable rates for a given distortion and code length. We extend the rate-distortion function using mutual f-information and provide new and strictly better bounds on achievable rates in the finite blocklength regime using super-modular f-divergences, and (iii) we provide a connection between the generalization error of algorithms with bounded input/output mutual f-information and a generalized rate-distortion problem. This connection allows us to bound the generalization error of learning algorithms using lower bounds on the f-rate-distortion function. Our bound is based on a new lower bound on the rate-distortion function that (for some examples) strictly improves over previously best-known bounds.

Laws of Large Numbers for Weighted Sums of Independent Random Variables: A Game of Mass

We consider weighted sums of independent random variables regulated by an increment sequence and provide operative conditions that ensure a strong law of large numbers for such sums in both the centred and non-centred case. The existing criteria for the strong law are either implicit or based on restrictions on the increment sequence. In our setup we allow for an arbitrary sequence of increments, possibly random, provided the random variables regulated by such increments satisfy some mild concentration conditions. In the non-centred case, convergence can be translated into the behaviour of a deterministic sequence and it becomes a game of mass when the expectation of the random variables is a function of the increment sizes. We identify various classes of increments and illustrate them with a variety of concrete examples.

Netolygūs įverčiai puasoninėje aproksimacijoje

Poisson approximation for the sum of independent random variables is investigates in this paper.

Upper-bound estimates for weighted sums satisfying Cramer’s condition

Let S = ω1S1 + ω2S2 + ⋯ + ωNSN. Here Sj is the sum of identically distributed random variables and ωj  > 0 denotes weight. We consider the case, when Sj  is the sum of independent random variables satisfying Cramer’s condition. Upper-bounds for the accuracy of compound Poisson first and second order approximations in uniformmetric are established.

On the Probabilities of Large Deviations of Combinatorial Sums of Independent Random Variables That Satisfy the Linnik Condition

On the Probabilities of Large Deviations of Combinatorial Sums of Independent Random Variables That Satisfy the Linnik Condition

On approximation and estimation of distribution function of sum of independent random variables

On approximation and estimation of distribution function of sum of independent random variables

Sharp concentration results for heavy-tailed distributions

Abstract We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $P(X>t) \leq{\text{ e}}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow \alpha \in [0, \infty )$ as $t \rightarrow \infty $. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of sub-Weibull random variables, but it can also produce new results for the sum of random variables with heavier tails. We show that the concentration inequalities we obtain are sharp enough to offer large deviation results for the sums of independent random variables as well. Our analyses which are based on standard truncation arguments simplify, unify and generalize the existing results on the concentration and large deviation of heavy-tailed random variables.

ON THE ACCURACY BE THE METRIC OF THE DENSITY ESTIMATION CONSTRUCTED BY DEPENDENT OBSERVATIONS

Solving many tasks in modern statistical research (such as studies of financial independence of investment and insurance companies, risk assessment of banking investments, assessment of indicators of financial stability of companies, etc.) requires consideration of dependent observations. Constructing estimates from dependent data is based on a rich historical experience of constructing estimates from independent data. To determine the accuracy of the constructed estimates, it is necessary to study the asymptotics of the sums of dependent random variables. In this process, the research methods of distribution of sums of independent random variables are extended to dependent random variables. Markov dependence is considered. It is one of the forms of weak dependence. Many authors see sums of random variables those joint distributions are determined by some "control" sequence of random elements. By I. Bokuchava, T. Shervashidze and Z. Kvatadze is are determined conditionally independent (Bokuchava I. V. (1984) Limit theorems for conditionally independent sequences. (in Russian) Teor. Verojatnost. i Primenen.-MathNet.Ru XXIX. (1984). №1,. pp. 192-193.1984. Theory of probability and its applications, 1985, 29:1, 190-196; Kvatadze Z., Shervashidze T. (2008) Some limit theorems for I.I.D. and conditionally independent random variables. The Second International Conference, "Problems of Cybernetics and Informatics". September 10-12, 2008. Baku, Azerbaijan. "Applied Stochastic Analysis" www.pci2008.science.az/4/12.pdf. Azerbaijan National Academy of Sciences. INSTITUTE OF INFORMATION TECHNOLOGY. Printing House of "Information Technology" Baku. 2008 . Vol. II. pp. 217-219) and chain dependent (I. V. Bokuchava, Z. Kvatadze, T. Shervashidze, On limit theorems for random vectors controlled by a Markov chain. Probability theory and mathematical statistics, vol. I (Vilnius, 1985), 231-250, VNU Sci. Press, Utrecht, 1987; Kvatadze Z., Sh. ervashidze T. (1987) On limit theorems for conditionally independent random variables controlled by a finite Markov chain. Probability theory and mathematical statistics. (Proc. 5th Japan-USSR Symposium on Probability Theory. Kyoto. 1986). Lecture Notes in Mathematics, 1299: 250-259. Springer-Verlag. Berlin (Germany)) дшьше distributions of normalized sums of random variables.  In the theory of non-parametric estimates, an important place is given to the estimation of the unknown density of the distribution. M. Rosenblatt and E. In the works of Parzen (M. Rosenblatt (1956), Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27, Chicago, 832-837, E. Parzen (1962), On estimation of a probability density function and mode . Ann. Math. Statist. 33, Stanford, USA, 1065-1076) a class of robust density estimates constructed from independent observations is considered. The accuracy of these estimates is known (E. (E. A. Nadaraya (1983), Nonparametric estimation of the probability density and regression curve. (in Russian) Tbilisi State Univ. Press, 1983) და  (Devroye L., Györfi L. Nonparametric density estimation: the  view. Wiley series in probability and mathematical statistics, Canada: John Wiley & Sons; 1985. p. 367)  by metrics. The study of the asymptotic behavior of the sums of dependent random variables made it possible to consider dependent observations in the theory of statistical estimates. Nonparametric density estimation and regression coefficient estimations with Markov chain observations are known (Yakowitz Sidney (1989) Nonparametric density and regression estimation for Markov sequences without mixing assumptions. 85721–Journal of Multivariate Analysis, 30: 124-136. Arizona, USA). The accuracy of the density estimation built with dependent observations is also determined by the metric (J. Meloche. Asymptotic behavior of the mean integrated squared error of kernel density estimators for dependent observations. Canadian Journal of Statistics. 2009., 18 (3): p. 205-211). Z. Kvatadze and B. Phardjiani constructed an accurate estimate of the density with conditionally independent and chain-dependent observations and determined their accuracy by a metric in the special case when the governing sequence takes two values ( ) (Z, Kvatadze, B. Phardjiani. On the Exactness of Distribution Density Estimates Constructed by Some Class of Dependent Observations. Mathematics and Statistics. 2019 Vol. 7(4), pp. 135-145. SAN JOSE). The accuracy of the heart estimation constructed by the chain-dependent estimations is also obtained in the metric case as well. In the present work, by Rosenblatt-Parzen-type core estimate of the density is constructed using conditionally independent observations. Are applied obtained by I. Bokuchava, T. Shervashidze and Z. Kvatadze results on the limit distribution of conditionally independent random sums of magnitudes and the accuracy of the constructed estimate are determined by the metric. T. is used during the proof of the theorem. Shervashidze and Z. The method discussed by Kvatadze. On the fixed  trajectory, the sum to be evaluated is expanded into the corresponding sums of the states of the control sequence. These totals are uncorrelated. The components of each sum on the fixed trajectory are independent. The dimensionality of ( ) functions defined on chain states is used (Kvatadze Z., Kvatadze TS. Limiting Distribution of a Sequence of Functions Defined on a Markov Chain. Proceedings of A. Razmadze Mathematical institute. Vol. 174. 2020. issue 2, 199-205) By fixing the trajectory obtained by partitioning the probability space with respect to the induced algebra. The applied method makes it possible to obtain the accuracy of other types of estimates constructed with dependent observations.

State-dependent importance sampling for estimating expectations of functionals of sums of independent random variables

Estimating the expectations of functionals applied to sums of random variables (RVs) is a well-known problem encountered in many challenging applications. Generally, closed-form expressions of these quantities are out of reach. A naive Monte Carlo simulation is an alternative approach. However, this method requires numerous samples for rare event problems. Therefore, it is paramount to use variance reduction techniques to develop fast and efficient estimation methods. In this work, we use importance sampling (IS), known for its efficiency in requiring fewer computations to achieve the same accuracy requirements. We propose a state-dependent IS scheme based on a stochastic optimal control formulation, where the control is dependent on state and time. We aim to calculate rare event quantities that could be written as an expectation of a functional of the sums of independent RVs. The proposed algorithm is generic and can be applied without restrictions on the univariate distributions of RVs or the functional applied to the sum. We apply this approach to the log-normal distribution to compute the left tail and cumulative distribution of the ratio of independent RVs. For each case, we numerically demonstrate that the proposed state-dependent IS algorithm compares favorably to most well-known estimators dealing with similar problems.

On probabilities of large deviations of combinatorial sums of independent random variables satisfying Linnik’s condition

We derive new results on asymptotic behaviour for probabilities of large deviations of combinatorial sums of independent random variables satisfying Linnik’s condition. We find zones in which these probabilities are equivalent to the tail of the standard normal law. The author earlier obtained such results under Bernstein’s condition. The truncations method is applied in proofs of results.

A Mutual Information Inequality and Some Applications

In this paper we derive an inequality relating linear combinations of mutual information between subsets of mutually independent random variables and an auxiliary random variable. One choice of a family of auxiliary variables leads to a new proof of a Stam-type inequality regarding the Fisher Information of sums of independent random variables. Another choice of a family of auxiliary random variables leads to new results as well as new proofs of results relating to strong data processing constants and maximal correlation between sums of independent random variables. Other results obtained include convexity of Kullback–Leibler divergence over a parameterized path along pairs of binomial and Poisson distributions, as well as a new duality-based argument relating the Stam-type inequality and entropy power inequality.

On asymptotic behaviour for probabilities of moderate deviations of combinatorial sums

We investigate an asymptotic behaviour for probabilities of moderate deviations of combinatorial sums of independent random variables having moments of order p > 2. We find zones in which these probabilities are equivalent to the tail of the standard normal law. The width of the zone is a function from the logarithm of a combinatorial variant for Lyapunov’s ratio. The author earlier obtained similar results under Bernstein’s and Linnik’s conditions. The truncations method is used in proofs of the results.

Sharp Rosenthal‐type inequalities for mixtures and log‐concave variables

AbstractWe obtain Rosenthal‐type inequalities with sharp constants for moments of sums of independent random variables which are mixtures of a fixed distribution. We also identify extremizers in log‐concave settings when the moments of summands are individually constrained.

Some Limit Theorems for Large Deviations of Sums of Independent Random Variables with Common Distribution Function from the Domain of Normal Attraction of a Stable Distribution

Some Limit Theorems for Large Deviations of Sums of Independent Random Variables with Common Distribution Function from the Domain of Normal Attraction of a Stable Distribution

From risk reduction to risk elimination by conditional mean risk sharing of independent losses

This paper studies diversification effects resulting from pooling insurance losses according to the risk allocation rule proposed by Denuit and Dhaene (2012). General comparison results are established for conditional expectations given sums of independent random variables. It is shown that these expectations decrease in the number of terms comprised in the conditioning sums. Additional inequalities are obtained under regression dependence in the sum. These general results are used to derive the monotonicity of the respective contributions of the participants with respect to the convex order, showing that increasing the number of participants is always beneficial under conditional mean risk sharing. New convergence results are obtained, showing that the variance of individual contributions tends to zero in many interesting cases. This provides actuaries with conditions ensuring that the risk can be fully eliminated within the pool, at the limit.